Neural representations for modeling variation in speech

• Neural acoustic models can be used to automatically model pronunciation variation. Pronunciation variation is best captured by intermediate layers of transformer models. Transformer-based embeddings capture details not expressed phonetic transcriptions. Variation in speech often quantified comparing transcriptions the same utterance. However, manually transcribing time-consuming and error prone. As an alternative, therefore, we investigate extraction from several self-supervised neural We use these representations compute word-based differences between non-native native speakers English, Norwegian dialect speakers. For comparison with earlier studies, evaluate how well match human perception them available judgements similarity. show that extracted a specific type (i.e. Transformers) lead better than two approaches on basis MFCC-based features. furthermore find features generally one middle hidden final layer. also demonstrate only segmental differences, but intonational durational cannot adequately represented set discrete symbols